To perform like the human visual system, imaging systems must automatically adapt to changing color casts in scene illumination. Simply put, white objects in a scene must be rendered as white, regardless of whether the scene illuminant was daylight, tungsten, fluorescent, or some other source. This process of automatic white adaptation is called “white balancing” and the corrective action determined by this adaptation mechanism is the white balance correction.
Automatic white balance algorithms employed in automatic printers, digital scanners, and digital cameras conventionally employ the digitized image information and related mathematical techniques to attempt to deduce from the image data the optimum level of white balance correction to be applied on a scene-by-scene basis to the image. It is known that errors in automatic white balance correction occur when the algorithm is unable to differentiate between an overall color cast caused by the scene illuminant and an overall color bias due to the composition of the scene. It is desirable, therefore, to be able to differentiate a color cast due to scene illumination from a color bias due to scene composition. It is also known that white balance errors occur due to color temperature variations within a class of scene illuminant. Late day direct sunlight imposes a yellowish color cast to a scene while skylight on a cloudy day will lend a bluish color cast to a scene. However, both lights are clearly daylight and will require substantially different white balance corrections. It is desirable, therefore, to also be able to account for scene illuminant color temperature variation when determining the white balance correction.
There are many methods described in the literature for determining the scene illuminant of a digital image. Some require special hardware at the time of image capture to make this determination. In commonly-assigned U.S. Pat. Nos. 4,827,119 and 5,037,198 a method of measuring scene illuminant temporal oscillations with the use of a dedicated sensor is described. Daylight will have no oscillation, while tungsten and fluorescent sources will fluctuate in output power due to the AC nature of their power supplies. The problem with any dedicated sensor approach is that it consists of two separate data collection and processing paths, one for illuminant detection and another for actual image capture. This leads to the potential of the dedicated sensor path losing synchronization and calibration with respect to the main image capture path. Additionally, the relatively limited amount of information captured by a dedicated sensor can severely limit the robustness of the scene illuminant determination. In commonly-assigned U.S. Pat. Nos. 5,644,358 and 5,659,357 the image data (video input) is combined with a luminance input to perform illuminant classification. (The nature of the luminance input is never described.) Rather than determining an overall illuminant for the scene, a low resolution version of the image is created and each image element (or “paxel”) within the low resolution image is individually classified into one of a number of possible scene illuminants. Statistics are performed on these paxel classifications to derive a best compromise white balance correction. The problem with this approach is that no explicit attempt is made to uncouple the effects of scene illuminant color cast from the effects of scene composition. Instead, a complex series of tests and data weighting schemes are applied after the paxel classifications to try and reduce subsequent algorithm errors. Japanese Patent JP2001211458 teaches a method very similar to that described in commonly-assigned U.S. Pat. Nos. 5,644,358 and 5,659,357, and has the same problems.
There are many methods described in the literature for determining a color temperature responsive white balance correction of a digital image. In commonly-assigned U.S. Pat. Nos. 5,185,658 and 5,298,980 a method of measuring the scene illuminant's relative amounts of red (R), green (G), and blue (B) power with dedicated sensors is described. The white balance correction values are derived from the ratios of R/G and B/G which are considered to be related to the color temperature of the scene illuminant. As with commonly-assigned U.S. Pat. Nos. 4,827,119 and 5,037,198, discussed above, the problem with any dedicated sensor approach is that it consists of two separate data collection and processing paths, one for illuminant detection and another for actual image capture, and these two paths can get “out of step” with each other. In the above referenced JP200121458 the illuminant classification step is further refined to represent a variety of subcategories within each illuminant class. In this way cooler and warmer color cast versions of the illuminant classes of daylight, tungsten, and fluorescent are determined. However, as stated before, there is no explicit method given for uncoupling illuminant color cast from scene composition variability and, as a result, a variety of involved statistical operations are required in an attempt to minimize algorithmic errors.
In commonly-assigned U.S. Pat. No. 6,133,983 Wheeler discloses a method for optical printing of setting the degree of color correction; i.e. a parameter used to determine the magnitude of applied color balancing to photographic images, based on camera meta-data. In particular, Wheeler discloses using the scene-specific measurements of the scene light level, camera-to-subject distance, flash fire signal, and flash return signal to classify an image as being captured either under daylight or non-daylight illuminant. It is stated that for images captured with daylight-balanced films there is no need to further distinguish the non-daylight illuminants because the same white balance correction methodology works regardless. As a result, commonly assigned U.S. Pat. No. 6,133,983 does not present a method for such subsequent illuminant discrimination. This approach fails when applied to imaging systems requiring further differentiation of non-daylight sources for accurate white balancing, or if any of the image metadata (i.e., scene light level, camera-to-subject distance, flash fire signal, and flash return signal) are corrupt or missing. In particular, the method disclosed by Wheeler requires the scene light level to be a measured quantity.
In conference paper “Usage of DSC meta tags in a general automatic image enhancement system” from the Proceedings of SPIE Vol. #4669, Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications III, Jan. 21, 2002, the authors Moser and Schroder describes a method of scene analysis regarding the likelihood of a photographic scene having been influenced by an artificial illuminant light source. The method disclosed by Moser and Schroder uses the camera meta-data of Fnumber (f) and exposure time (t) to calculate a “pseudo” energy (pE) quantity for a digital image derived from a digital camera using the formula:
  pE  =            ln      ⁡              (                  t                      f            2                          )              .  Moser and Schroder then use the pE quantity to analyze digital images with regard to the likelihood of the scene illumination source and corresponding resultant color cast. While the method disclosed by Moser and Schroder is useful for analyzing digital images, as disclosed it is not accurate enough to produce consistent automatic white balance correction results for a practical digital enhancement system. This is principally due to the inherent relative, as opposed to absolute, nature of the “pseudo” energy quantity. Two digital cameras with substantially different energy requirements for producing acceptable images will have substantially different “psuedo” energy values for the same scene illumination conditions. Similarly, these same two digital cameras can produce identical “psuedo” energy values when producing digital images with substantially different scene illumination sources.